ABSTRACT
Hand hygiene is critical for declining the spread of viruses and diseases. Over recent years, it has been globally known as one of the most effective ways against COVID-19 outbreak. The World Health Organization (WHO) has suggested a 12-step guideline for hand rubbing. Due to the importance of this guideline, several studies have been conducted to measure compliance with it using Computer Vision. However, almost all of them are based on processing single images as input, referred to as baseline models in this paper. This study proposes a sequence model in order to process sequences of consecutive images as input. The model is a mixture of Inception-ResNet architecture for spatial feature extraction and LSTM for detecting time-series information. After training the model on a comprehensive dataset, an accuracy of 98.99% was achieved on the test set. Compared to the best baseline models, the proposed sequence model is correspondingly about 1% and 4% better in terms of accuracy and confidence, though 3 times slower in inference time. Furthermore, this study demonstrates that the accuracy metric is not necessarily adequate to compare different models and optimize their hyperparameters. Accordingly, the Feature-Based Confidence Metric was utilized in order to provide a more pleasing comparison to discriminate the proposed sequence model with the best baseline model and optimize its hyperparameters.
ABSTRACT
Hand hygiene is critical for declining the spread of viruses and diseases. Over recent years, it has been globally known as one of the most effective ways against COVID-19 outbreak. The World Health Organization (WHO) has suggested a 12-step guideline for hand rubbing. Due to the importance of this guideline, several studies have been conducted to measure compliance with it using Computer Vision. However, almost all of them are based on processing single images as input, referred to as baseline models in this paper. This study proposes a sequence model in order to process sequences of consecutive images as input. The model is a mixture of Inception-ResNet architecture for spatial feature extraction and LSTM for detecting time-series information. After training the model on a comprehensive dataset, an accuracy of 98.99% was achieved on the test set. Compared to the best baseline models, the proposed sequence model is correspondingly about 1% and 4% better in terms of accuracy and confidence, though 3 times slower in inference time. Furthermore, this study demonstrates that the accuracy metric is not necessarily adequate to compare different models and optimize their hyperparameters. Accordingly, the Feature-Based Confidence Metric was utilized in order to provide a more pleasing comparison to discriminate the proposed sequence model with the best baseline model and optimize its hyperparameters.
ABSTRACT
Hand hygiene plays a crucial role in healthcare environments which can cease infections and diseases from spreading. It is also regarded as the second most effective way to control the transmission of COVID-19. The World Health Organization (WHO) recommends a 12-step guideline for alcohol-based hand rubbing. Compliance with this guideline is vital in order to clean the hands thoroughly. Hence, an automated system can help to improve the quality of this procedure. In this study, a large-scale and diverse dataset for both real and fake hand rubbing motions is collected as the first stage of building a reliable hand hygiene system. In the next stage, various pre-trained networks were analyzed and compared using a swift version of the Separation Index (SI) method. The proposed Swift SI method facilitates choosing the best pre-trained network without fine-tuning them on the whole dataset. Accordingly, the Inception-ResNet architecture achieved the highest SI among Inception, ResNet, Xception, and MobileNet networks. Fine-tuning the Inception-ResNet model led to an accuracy of 98% on the test dataset, which is the highest score in the literature. Therefore, from the proposed approach, a lightweight version of this model with fewer layers but almost the same accuracy is produced and examined. In the final stage, a novel metric, called Feature-Based Confidence (FBC), is devised for estimating the confidence of models in prediction. The proposed confidence measure is able to profoundly differentiate models with similar accuracy and determine the superior one. Based on the metrics results, the Inception-ResNet model is about 2x slower but 5% more confident than its lightweight version. Putting all together, by addressing the real-time application concerns, a Deep Learning based method is offered to qualify the hand rubbing process. The model is also employed in a commercial machine, called DeepHARTS, to estimate the quality of the hand rubbing procedure in different organizations and healthcare environments.